Papers with IWSLT translation tasks
Multiscale Collaborative Deep Models for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Neural machine translation models with deeper neural networks are difficult to train. |
| Approach: | They propose a MultiScale Collaborative framework to boost gradient back-propagation . they let each encoder block learn a fine-grained representation and enhance it . |
| Outcome: | The proposed framework outperforms baseline models on translation tasks with three translation directions and achieves a BLEU score of 30.56 on the English-to-German task. |
Improving Deep Transformer with Depth-Scaled Initialization and Merged Attention (D19-1)
Copied to clipboard
| Challenge: | Existing methods to improve NLP convergence and computational overhead are limited by stacking more layers. |
| Approach: | They propose a depth-scaled initialization method which reduces parameter variance at initialization and reduces output variance of residual connections to ease gradient back-propagation. |
| Outcome: | The proposed method outperforms the base model on translation tasks with five translation directions while matching the decoding speed of the baseline model. |
Differentiable Sampling with Flexible Reference Word Order for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to correct exposure bias in machine translation are inadequate . scheduled sampling assumes that words are aligned at each time step . |
| Approach: | They propose a differentiable sampling algorithm that optimizes the probability that the reference can be aligned with the sampled output. |
| Outcome: | The proposed approach improves BLEU on translation tasks and is simpler to train with no sampling schedule. |